Statistical Science 

2011. Vol. 26, No. 3, 329-331 

DOI: 10.1214/11-STS352REJ 

Main article DOI: 10. 1214/11-STS352 

(^^^stitu^: ol Mathematical Statistics, 2011 

Rejoinder 

D. A. S. Fraser 



(N 

o 

(N 
C 

m 



-4-* ■ 

ctf - 
^— > . 

> ' 



q 
o 

(N 



X 



1. INTRODUCTION 

Very sincere thanks to the discussants for choosing 
to enter a virtual minefield of disagreement in the 
development history of statistics. For we just need 
to recall the remark that fiducial is Fisher's "biggest 
blunder" and place it alongside the fact that fidu- 
cial was the initial step toward confidence, which 
arguably is the most substantive ingredient in mod- 
ern model-based theory: the two differ in minor de- 
velopmental detail, with fiducial offering a probabil- 
ity distribution as does Bayes and with confidence 
offering just probabilities for intervals and special 
regions. Statistics has spent far more time attack- 
ing incremental steps than it has seeking insightful 
resolutions. 

As a modern discipline statistics has inherited two 
prominent approaches to the analysis of models with 
data; of course such is not all of statistics but is 
a critical portion that influences the discipline widely. 
How can a discipline, central to science and to crit- 
ical thinking, have two methodologies, two logics, 
two approaches that frequently give substantially 
different answers to the same problems. Any astute 
person from outside would say, "Why don't they put 
their house in order?" And any serious mathemati- 
cian would surely ask how you could use a lemma 
with one premise missing by making up an ingredi- 
ent and thinking that the conclusions of the lemma 
were still available. Of course, the two approaches 
have been around since 1763 and 1930 with regular 
disagreement and yet no sense of urgency to clarify 
the conflicts. And now even a tired discipline can 
just ask, "Who wants to face those old questions?": 
a fully understandable reaction! But is complacency 
in the face of contradiction acceptable for a central 
discipline of science? 
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A statistical model differs from a deterministic 
model in having added probability structure that 
describes the variability typically present in most 
applications. So, in an application with a statistical 
model and related data it would then seem quite 
natural that that variability would enter the con- 
clusions concerning the unknowns in an application: 
what do I know deterministically, and what do I 
know probabilistically? 

And that is what Bayes proposed in 1763: prob- 
ability statements concerning the unknowns of an 
investigation. Many have had doubts and said there 
was no merit in the proposal; and many have ac- 
ceded and became strong believers. And then Fisher 
(1930) also offered probabilities concerning the un- 
knowns of an investigation, but by a different argu- 
ment, and the turf fight began! Bayes had hesitantly 
examined a special problem and added a random 
generator for the unknown parameter, and Fisher 
had worked more generally and used just the ran- 
domness that had generated the data itself. 

But then a third person, Lindley (1958), from the 
same country said that the second person, Fisher, 
couldn't use the term probability for the unknowns 
in an investigation, as the term was already taken 
by the first person, Bayes. And strangely the disci- 
pline complied! Decades went by and anecdotes were 
traded and things were often vitriolic. 

2. WHAT DOES THE ORACLE SAY? 

Consider some regular statistical model f(y,0), 
together with a lower ^-confidence bound Op(y), and 
also a lower ^-posterior bound 9{y) based on a prior 
tt(9): What does the oracle see concerning the us- 
age of these bounds? He can investigate any long 
sequence of usages of the model, and He would have 
available the data values yi and of course the preced- 
ing parameter values Q{ that produced the yi values; 
He would thus have access to {(6i,yi) ■ i = 1, 2, . . .}. 

First consider the lower confidence bound. The or- 
acle knows whether or not the Oi is in the confidence 
interval {Op(yi), oo), and He can examine the long- 
run proportion of true statements among the asser- 
tions that Oi is in the confidence interval {Qp{yi), oo), 
and He can see whether the confidence claim of a /3- 
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proportion true is correct. In agreement with the 
mathematics of confidence, that proportion is just f3. 

Now consider the lower posterior bound. The or- 
acle knows whether 8i is in the posterior interval 
(9s(yi), oo), and He can examine the long-run pro- 
portion of true statements that 9i is in the poste- 
rior interval (0g(?/j), oo). Now suppose the long-run 
pattern of 9i values just happened to correspond to 
the pattern 7r(0); then in full agreement with the 
mathematics of the Bayes calculation, the Oracle 
would see that long-run proportion of true state- 
ments among assertions that 9i is in the posterior in- 
terval (#g(?/j), oo) was correct, was just the stated f3. 

But what if the long-run pattern of 6i values was 
different from the introduced tt(0) pattern? Then in 
wide generality the long-run proportion of true sta- 
tements among assertions that Oi is in the posterior 
interval (9p(yi), oo) would not be j3\ In other words, 
the confidence procedure is always right, and the 
Bayes procedure is typically wrong, unless the prior 
was guessed correctly. Seems like a poor trade-off! 

Now consider further what a prior actually does 
in producing parameter bounds or quantiles that 
are different from the confidence bound. Prom an 
asymptotic viewpoint a prior can be expanded as 
exp(a0/n 1 ^ 2 + c9 2 /n) to the third order, as men- 
tioned but not pursued in Section 6(iv). This pro- 
vides a direct displacement of the confidence bound 
in standardized units and produces an 0(l)-shift 
away from the claimed (3 value, either up or down 
depending on the sign of a\ Hardly an argument for 
using the Bayes procedure unless there was some 
very urgent need for a quick and dirty calculation. 

3. RESPONSE TO THE DISCUSSANTS 

Christian Robert 

Christian presents a very committed Bayes view- 
point and quite correctly admonishes me for not dis- 
tinguishing what Thomas Bayes did and what has 
followed in the same theme. But going beyond the 
minor detail, Bayes added a distribution for a pa- 
rameter, a distribution that was not part of the bi- 
nomial example under consideration and then used 
that distribution for probability analysis. And much 
of modern Bayesian statistics does precisely that: in- 
troduces an artifact distribution for expediency or 
convenience and then works reassuringly within ac- 
cepted probability calculus. Indeed, this is the pri- 
mary theme of the article: adding something arbi- 
trary gives something arbitrary no matter how at- 
tractive the material labeled probability might or 
might not be, or no matter what might be available 



by other methods of analysis. If one faces a probabi- 
lity-type claim, it is fair enough to simulate and eval- 
uate the claim, and that is what coverage probability 
is all about, as the invincible Oracle well knows. 

The marginalization paradoxes do appear in the 
literature but are widely neglected and not "exten- 
sively discussed" as Christian suggests. They apply 
to any proposal for a distribution to describe an un- 
known vector parameter, whether obtained by the 
Bayes inversion of a density or the frequentist in- 
version of a pivot, such as fiducial, confidence struc- 
tural or other. There is an immutable contradiction 
built into the hope to describe a vector parameter 
by a distribution. Curvature of an interest parame- 
ter has emerged as the critical source for this con- 
tradiction. Take a bivariate parameter, a data point 
and an interest parameter value: if the parameter 
is linear, the confidence and the Bayes values are 
equal; if then parameter curvature is introduced, we 
have that the confidence value and the Bayes value 
change in opposite directions! One has the cover- 
age property and the "other" acquires bias at twice 
the rate of the departure from linearity. And the 
"other" uses the name probability with an assertive- 
ness coming from the use of the probability calculus, 
conveniently overlooking that an artifact was intro- 
duced in place of the input needed for the validity 
of the probability calculus for the application. 

Maybe it is time to address the Pandora's box and 
check for a Madoff pyramid: too good to be true. 

Larry Wasserman 

Larry presents a pragmatic view of the Bayes ap- 
proach, acknowledging its rich flexibility but recom- 
mending coverage cautions. His five examples are 
most welcome concerning the wider spheres of ap- 
plication and he is to be complemented on the skill- 
ful innovations. I do quarrel, however, with his rein- 
forcement of personality cults in statistics. It seems 
that statistics has suffered greatly from this exter- 
nalization of the scientific method, as if there were 
different flavors of scientific thinking and mathemat- 
ical logic and that these might gain concreteness 
when personalized. 

Kesar Singh and Minge Xie 

Confidence for estimation and exploration? It is 
deeply unfortunate that statistics chooses at many 
steps to malign its major innovators, for example, 
Fisher with his "biggest blunder" referent for 
the initiative that gave us confidence. What Fisher 
didn't do was present his major innovations in a fully 
packaged form ready to withstand a few centuries of 
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challenges and modification: What? We still have to 
do a little bit of thinking! Tough! He clearly must 
generously have expected others to have his insight 
and wisdom! 

Fiducial, confidence, structural or other? It is just 
pivot inversion with variation in context, conditions 
or interpretation: the big risk was described by Da- 
wid, Stone and Zidek (1973) and curvature is now 
identified as the prime cause. To have different na- 
mes to fine tune for different applications or different 
explorations would seem to take emphasis away from 
the proper calibration of the tool, as the primary 
concern for most applications. 

Statistics routinely combines likelihoods as appro- 
priate, so it is not correct to attribute this to Baye- 
sian learning; perhaps the central sectors of statis- 
tics were just slow to glamorize the good things in 
their statistical modeling. Putting a prior on a like- 
lihood is a different operation downstream from as- 
sembling the likelihood in the relevant broader con- 
text, although it does seem convenient for the Bayes 
approach to co-opt it as their own contribution when 
it was somewhat neglected by the "others." 

Tong Zhang 

Where does the pivot come from? Fisher's de- 
velopment of confidence or whatever attracted the 
mathematicians' criticisms, mostly because it wasn't 
proposed in a fully developed form. It was then shred- 
ded, fully ignoring the emerging recognition of its 
innovative genius. Certainly the need to clarify the 
origin of the key ingredient, "the pivot," is of fun- 
damental importance, as Tong suggests: using all 
the data in an appropriately balanced way, respect- 
ing continuity and parameter direction from data, 
and more. Whether these should be bundled under 
a term optimality may be questionable, but doesn't 
diminish the importance of the individual criteria; 
for some recent emphasis on continuity see Fraser, 
Fraser and Staicu (2010). 

4. SOME CONCLUDING INVOCATIVE 
REMARKS 

An inference distribution for a vector parameter 
is inherently a contradiction. Information from two 



different sources can be reported separately, with 
combination not by principle. Combining likelihoods 
is a consequence of combining models, typically fol- 
lowing from independence; the Bayes claim that it 
comes from the use of the Bayes argument is after 
the fact and disingenuous. Inverting a density and 
inverting a pivot are different except in the linear 
case, but the first can sometimes approximate the 
second. 

The question was asked: "Is Bayes posterior just 
quick and dirty confidence?" And the case was made 
for "Yes": Bayes posterior is just quick and dirty 
confidence: quick in the sense of easier than using 
quantiles to determine how 6 affects data; and ap- 
proximate in the sense of a wide spread need to use 
approximation methods. 

Not everyone liked the blunt question. One dis- 
cussion expresses discomfort with such a direct con- 
frontation to the Bayes approach; one discussion 
adds additional support examples; and the two re- 
maining discussions speak more to methods and mod- 
ifications of confidence distributions, overlooking the 
risks. But no one argued that the use of the condi- 
tional probability lemma with an imaginary input 
had powers beyond confidence, supernatural pow- 
ers. 
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